home
***
CD-ROM
|
disk
|
FTP
|
other
***
search
/
HPAVC
/
HPAVC CD-ROM.iso
/
NEOCOG10.ZIP
/
NEOC.DOC
< prev
next >
Wrap
Text File
|
1994-07-15
|
28KB
|
619 lines
────────────────────────────────────────────────────────────
USER'S GUIDE
NeoC Explorer Version 1.0
Copyright 1994 (C), Szabolcs Szakacsits
Jozsef Attila University, Szeged, Hungary
July 22, 1994
────────────────────────────────────────────────────────────
1. Introduction to NeoC Explorer
─────────────────────────────────
The NeoC program is an implementation of Fukushima's
Neocognitron neural network. It's purpose is to test the model, to
facilitate interactivity for the experiments, and to make the
Neocognitron neural net operation more understandable.
Some substantial features:
■ friendly graphic user interface
■ explorer and tester operation modes
■ easy neural net construction
■ graphic or numerical displaying of cells and weights
■ training by patterns or modules step by step, or
complete neural net training
■ displaying of the patterns and recognition results
■ recognition statistics
■ even more statistics of the tests in a .LOG file
during tester mode
The minimal system requirements to run NeoC are a 286 AT
compatible computer, a VGA video card, and a mouse. A fast
computer with a mathematical coprocessor is recommended because of
the many computations.
There are two utility programs for NeoC. One of them is the MAP
(Make Pattern). Using it one can design new patterns easily and
quickly in the graphic mode with mouse support. The other program
- 1 -
is the DIP (Display Pattern). It converts the binary pattern files
to the standard output as text so that the text file can be
interfaced with the user's own software. One can use this text
file for one's own program (probably, the text file will have to
be modified somewhat to conform to the user's own file type), and
in this way the experiments can be done with the same patterns
comparing NeoC and other neural nets. For this purpose, another
solution can be found in the MAP.DOC file.
NeoC uses three main type file formats. They specify the
■ neural net structure and parameters (files with
extension .ndf)
■ complete neural net (files with extension .net)
■ train and input patterns (files with extension .ptt)
The first two are text files, so one can create them using a
word processor. However I suggest doing this with NeoC because
it's much easier. The structure of a .ndf file is, for instance,
the following:
2
1 8 1
6 6 3
6 4 3
3 2 3
3 1 2
0.100000
10
1.000000 1.000000
99.000000 99.000000
The first line is the number of net modules, the next lines are
the U0 (input layer), US1 (S-layer of module 1), UC1 (C-layer of
module 1), US2 (S-layer of module 2) and UC2 (C-layer of module
2). These lines consist of three columns, i. e. the number of
planes in that layer, the size of planes in the current layer (eg.
the above second line means that the input layer has one plane
with 8 x 8 cells), and the mask size (reception field). Note the
size (N) means actually a real size N x N field. The following
lines are the alpha parameter, the decision parameter (see later),
the r parameters in each and every module, and the last line that
contains the q parameters.
The .ptt binary files are created by the MAP program.
The NeoC command line structure is shown below.
NeoC [-m] [file1 [file2 [file3]]]
The file1 is a neural net definition file (.ndf), file2 is a train
file (.ptt), and file3 is an input file (also a .ptt file). If the
command line does not contain any parameters (except for the -m
option), the program will suggest a net structure, and it may be
modified and saved later. If one tests a particular net structure
- 2 -
with the same train and input file regularly, then it is useful to
make a batch file which consists of the above described lines, and
to run it. If only a monochrome monitor is available, then one
should run NeoC with the -m option.
Information can be obtained on command line structure, if the
?, -?, h, or -h options are used.
Usage of NeoC consists of six components: Main Menu, 'Explorer
Panel', 'Net Input' and 'Identify as', NeoCognitron Neural Net,
Structure, and 'Tester Panel' windows.
2. Main Menu
─────────────
Main Menu structure of NeoC:
■ Structure
■ Load
■ Save
■ Change
■ Net
■ Load
■ Save
■ Reset
■ Operation
■ Explorer
■ Tester
■ DOS
■ DOS shell
■ Exit
■ Help
■ NeoC Explorer
■ About
The neural net structure cannot be changed in the NeoCognitron
Neural Net window because of dynamic memory allocation of the net
structure. If one desires to change it, then one has to use the
Structure/Change menu.
The Net/Reset menu can be used if one is in the Explorer mode
(it is automatic in the Tester mode). By selecting it, one may
reset the weights of the current module or every module. The value
of the weights will be set between 0 and 1 randomly.
The 'Explorer Panel' window will appear if one uses the
Operation/Explorer menu (this is the default mode). Here are the
most possibilities to manipulate and examine the net.
The 'Tester Panel' window will appear instead of the 'Explorer
Panel' if the Operation/Tester menu is used. Here, the net
performance can be measured and analyzed.
Some of the menus will not always make sense, should this be
the case an information message will be displayed on the screen.
- 3 -
3. Explorer Panel
──────────────────
The white texts are static and the red ones are dynamic in the
windows and window fields. The red texts can be given a new value
by pointing and clicking on them with the left, or right, or both
buttons. The left button on the mouse will increase the value by
one, the right button will decrease the value by one, and pushing
both at the same time will cause a small window to appear on the
screen. The new value may be written into it. This method works in
the case of text variables also. The boxes in view may be left by
pressing the ESC key or by pushing both of the mouse buttons at
the same time if the new value given is not desired. If the new
value isn't correct, a message will indicate the error.
The 'Train file' and 'Input file' fields determine which
pattern files will be trained and recognized. These fields must be
filled in, and the file types must be in the .ptt file format. The
pattern size, in both files, and the input layer size of the net
must be equal. If these conditions aren't granted, the program
will display an error message. The numbers after the file names,
are the number of patterns in the current (train or input) file.
The net weights and planes (ie. the sets of cells) can be
viewed by using the 'VIEW' button which is on the bottom right
part of the 'Explorer Panel' window. The 'Display type' field can
be:
■ Graphic
■ Numerical
The fields which are below the 'Display type' field, determine
what kind of elements will be displayed (S-planes, C-planes,
weights). If a trained net exists, choose a pattern from the train
or input file in the 'Net Input' window by using the 'PAGE'
button, and click on the 'VIEW' by using the mouse button. At this
time the current values of the net cells and/or weights will be
calculated and afterwards displayed. When using the numerical
display mode, one can page forward by clicking with the mouse
button or one can exit with the ESC key. When using the graphic
mode, the solid yellow squares mean positive activity of the cells
in the S-planes or C-planes, and they mean trained weights in the
reception fields, i.e. the learned masks are shown.
The NeoC program provides information on the recognition in the
bottom right corner of the Explorer Panel window. The +, -, ?
columns mean the percentage of correctly, incorrectly and
undecided recognized input patterns. In this version of NeoC, a
pattern recognition is undecided if there is no active output or
there are more than one active outputs.
Since there is no explicit determination of the pattern
categories in the .ptt file one has to follow the rules shown
below exactly, if the proper recognition statistics are requested:
- 4 -
■ If the number of train patterns is N (they will be
treated as different category patterns), then the
number of input patterns has to be M x N , where
M >= 1 .
■ One has to have M patterns from each train pattern
category in the input file.
■ The order of categories has to be the same in both
files (in the train file and in the input file).
For example, if the structure and the pattern categories of a
train file are XYZ (X,Y and Z mean a pattern and category
simultaneously), then the structure of an input file can be XYZ,
or XXYYZZ, etc. It's not necessary to do this if the correct
statistic is not desired because a recognition result can be
viewed in the 'Identify as' window by using the 'PAGE' button in
the 'Net Input' window. The left mouse button will page forward,
while the right one will page backwards.
The 'Train type' field in the right top corner of the Explorer
Panel determines the type of the train. These types can be:
■ PATTERN
■ MODULE
■ NET
These types will be discusses at the description of the TRAIN
button. The field can be changed by using one of the mouse
buttons.
The 'Net input' field under the 'Train type' field, specifies
either, what the net will learn when the 'TRAIN' button is
selected or the pattern type (train or input) through which the
information on the net will be displayed when the 'VIEW' button is
used. The values of this field may be:
■ TRAIN PTT (train pattern)
■ TRAIN FILE
■ INPUT PTT (input pattern)
■ INPUT FILE
By changing this field then one can see either one pattern of
the train file or one pattern of the input file in the 'Net Input'
window. There is a difference only in that case between 'TRAIN
FILE' and 'TRAIN PTT', and between 'INPUT FILE' and 'INPUT PTT' if
the 'RECOGNIZE' button is used. At this time, the net will try to
recognize either all the patterns of the current file successively
(in case of 'TRAIN FILE' or 'INPUT FILE'), or only the current
pattern that is situated in the 'Net Input' window (if the 'Net
input' field is 'INPUT PTT' or 'TRAIN PTT')
One can give the number of the epoch of a pattern at the 'Num.
of epoches' field. This value is valid in all the modules. Since
the Neocognitron learns module by module, the number of the
- 5 -
pattern presentations is equal to the number of modules multiplied
by the value of 'Num. of epoches' field. If q values are large
(e.g. q1=100, q2=10000), then the recommended value of this field
is
MAX( ROUND_UP( |S-planes in module i| / |train patterns| ) )
where i is 1,2, ...to the last module. If one uses smaller q
values and a bigger epoch number, then one will get the same
result but more slowly because this implementation is a supervised
neural net although it can also work in an unsupervised mode with
a little modification in the source.
The 'Current module' field is important when one activates the
'VIEW' or 'TRAIN' buttons. This field determines the current
module if the net train is started and the 'Train type' field is
'PATTERN' or 'MODULE'. If the 'Train type' field is 'NET', then
this field does not have an influential effect during the training
because the train is automatic. If one uses the 'VIEW' button,
then this field specifies what module (ie. its S-planes, C-planes
and weights) will be displayed with regard the pattern in the 'Net
Input' window.
Information will be displayed about the values of some part of
the net graphically or numerically by using the 'VIEW' button. The
display is influenced by the alpha parameter, the decision
parameter, the r parameters, the pattern in the 'Net Input' window
and the 'Current module' field. The mouse cursor will disappear
during the computing. The process can be canceled by hitting the
ESC key on the keyboard. Finishing this action, the program will
show the required information relating to the 'Display type'
fields.
The 'TRAIN' button. The learning done by the net can be done in
three ways. These ways are the net learning by patterns, net
learning by modules (from module to module) or full net learning.
In the first case (train by patterns), the 'Net input' field has
to be set to 'TRAIN FILE' or 'TRAIN PTT', and the pattern, that it
is desired that the net learn, has to selected and placed in the
'Net Input' window. These settings are automatic in the other two
cases. What is important before the net can start to train in the
three different cases?
1. Training by patterns: The alpha, r and q parameters, the
S-column, the pattern in 'Net Input' window, the
'Display type' fields, the 'Current module' and the
'Num. of epoches' field are important. The mouse cursor
will also disappear during the computing. The process
can be canceled by hitting the ESC key on the keyboard.
This cancellation method is valid in the next two cases
too.
2. Training by modules: The same things must be set-up as
above except for the 'Display type' fields and the 'Net
Input' window. In this kind of training, these two
- 6 -
elements have no meaning because all of the patterns
will be trained.
3. Training of the complete net: Just the 'Num. of epoches'
field and the parameters of the 'NeoCognitron Neural
Net' window are important exclusive of the decision
parameter.
If the 'RECOGNIZE' button is used, then the program will
categorize either the current pattern (situated in the 'Net
Input') or each and every pattern file in the current .ptt file.
Before recognition, one can set in the 'Net input' field what is
desired to be categorized: the train file (TRAIN FILE), a train
pattern (TRAIN PTT), the input file (INPUT FILE) or an input
pattern (INPUT PTT). After a training session, it is worth while,
to make sure of that the train file was able to teach the net:
that the net learned all of the train patterns correctly. In the
next step, one can try to make the net recognize the input
patterns. If the 'Net input' field is 'INPUT PTT' or 'TRAIN PTT',
then the net tries to recognize the pattern that is situated in
the 'Net Input' window. This may be necessary if one had to alter
the net parameters because the answer of the net was incorrect for
that pattern in the former recognition attempt. This may or may
not be a desired final step. It depends upon the curiosity of the
user.
4. 'Net Input' and 'Identify as' windows
─────────────────────────────────────────
One can view the patterns of the train file or input file in
the 'Net Input' window which is related to the 'Net input' field
which is situated in the 'Explorer Panel' window.
1. By clicking with the left or right mouse button on the
'PAGE' button at the bottom of the 'Net Input' window,
the user will cause a train or input pattern to be
displayed in the window, and the results of the previous
recognition attempt will appear in the 'Identify as'
window.
2. If the 'Identify as' window is empty, then either there
was no active cell in the last C-layer of the net, i.e.
the net couldn't identify the pattern is situated in the
'Net Input' window, or the result of the former
recognition was cancelled by the effect of another event
(e.g. new training, modification of the 'Net input'
field, loading of a new .ptt file, etc.).
3. If a 'big solid square' appear in the 'Identify as'
window, this means that, there were more than one active
cells in the output layer of the net.
- 7 -
5. NeoCognitron Neural Net window
──────────────────────────────────
The white texts are static, and the red ones are dynamic here
too. See the treatment of these items above (first part of
Explorer Panel). In the left column, are the names of the net
layers i.e. U0 is the input layer, US1 is the S-layer of the
module 1, UC1 is the C-layer of the module 1, etc. Features of the
layers are in their lines successively. Specifically
■ The number of the planes
■ The size of the plane
■ The size of the mask (reception field)
■ The size of the S-column
■ The r (sensitivity) parameter
■ The q (learning) parameter
The first three features can be modified in only that case if
the Structure/Change menu is called.
I found that the size of the S-column was not too significant.
I recommend small values (2-4).
The role of the r (sensitivity) parameter is very important. It
influences the result of the comparison between the reception
fields of the S-cells and the learned mask. If this value is small
(e.g. below 2 depending on the size of the mask), then the net is
more tolerant of the pattern differences. If the r parameter is
large (e.g. above 8), then the net can identify only the learned
mask in the reception field, and it sets to zero the activity of
the other S-cells. One of the experiment subjects can be the
determination of the optimal r parameters.
The q parameter controls the speed of the train. If this value
is large then the learning is faster. In this case, the net will
learn a part of the patterns otherwise it can learn a pattern that
doesn't exist in any pattern (e.g. a combination of a part of two
or more patterns). I suggest big values (q1=100 in the first
module, q=10000 in the others) in this version of NeoC. The first
version of NeoC was the unsupervised model but the result of the
learning was difficult to evaluate.
One can tune the tolerance of pattern deformation of the
patterns by using the alpha parameter. I have detected no effect
so far. In the most cases, I used the 0.01 value.
A train pattern category can be elicited from the net by the
decision parameter if the net was not able to categorize a pattern
at the first identification attempt (no active cell or more active
cells than one in the last layer). If its value is 0, then it
doesn't influence the recognition (original Neocognitron model).
The recommended value is between 0 and 20.
- 8 -
6. Structure window
────────────────────
This window can be called up by the Structure/Change menu. One
is able to set-up the net structure here. The fields mean the same
as in the 'NeoCognitron Neural Net' window.
The value of the 'Num. of modules' field can be between 2 and
5.
One must invariantly follow the next rule. The 'Plane number'
field of the C-layer of the last module must be equal to the
number of the train patterns. The first output cell will identify
the first train pattern category. The second one will identify the
second train pattern category, e.t.c.
Recommended but not obligatory: the 'Plane number' of the
module i has to be a multiple of the number of train patterns. For
example, if there are ten train patterns, then the 'Plane number'
of the module i can be 10, 20, etc. If large values are set, then
the net will work more slowly. If small values are set, then the
net won't learn uniformly from each and every train pattern.
The values of the 'Plane size' field must be reduced layer by
layer from the input layer U0 to the last layer. The 'Plane size'
is always 1 x 1 in the last layer.
The 'Mask size' is optimal when it is equal to the 'Table size'
of the previous layer minus the 'Table size' of the current layer
plus one. In this case the weights of the cells of the current
layer cover exactly the planes of the previous layer. Of course,
this field can be smaller or larger.
7. Tester Panel
────────────────
The 'Tester Panel' window can be selected by the
Operation/Tester menu. This work mode is the most complex and
efficient part of the NeoC program. One can test a previously
created net construction. The neural net initialization, learning
and recognition will happen continuously and automatically as long
as neither the number of the tests reach the value specified in
the 'Num. of tests' field or the ESC key is hit.
The meaning and managing of the train and input file,
preparation of the right statistics, and the meaning of the 'Num.
of epoches' field are the same as in the 'Explorer Panel' window.
A logbook file (recommended extension is .log) will be made
during the time that the program is running. If this file is
already in existence, then further information will be appended to
it. The logbook file reports the recognition in each test, the
recognition percentage of every input pattern, and recognition
percentage of input patterns that are in the same category. The
logbook file name can be added in the '.LOG file' field.
- 9 -
A discussion of the structure of a .log file is next. The net
parameters are the first part of the .log file. The pattern
categories ( 0 1 2 etc.), i.e. the train patterns are between the
lines that are made with the character '-'. Below the line, the
result of the recognition test follows on every line. At the end
of lines, the '*' means the current best performance while the '='
means a performance result equal to the best performance. The
pattern categories ( 0 1 2 etc.) are again under the next line.
The recognition percentage of every input pattern proceed as far
as the line composed of the character '='. The last table line
contains the recognition percentages of the same category patterns
and the average performance.
Not only a .log file is made but a neural net is stored which
has given the best performance. The neural net name is determined
by the '.NET file' field. Thus one can load these net in the
'Explorer Panel' window and view the NeoC results after a
recognition process.
8. Epilog
──────────
This software is FREEWARE. It may be freely duplicated and
distributed. It was created for education and research purposes.
A few words about the new versions:
■ running under Windows 3.1 operation system
■ pattern categories => evaluation of tests will always
be possible
■ add unsupervised learning
■ add built and designed masks
■ new transfer functions
If you find NeoC Explorer useful or interesting and you are
interested in the next versions, then please register it. Your
registration will contain NeoC and also MAP. You can do so by
sending a small donation to my address, and in this way you will
support updates of production, and will receive the next upgrades.
See the details in the REGISTER.TXT file.
If you have any questions, comments or suggestions, please feel
free to write to me.
ADDRESS: Szakacsits Szabolcs
Kodaly 2. 2. lph.
H-8800 Nagykanizsa
HUNGARY
E-MAIL: jzp0119@huszeg11.bitnet
jzp0119@vm.cc.u-szeged.hu
jzp0119@alfa.cab.jate.u-szeged.hu
FAX: +36-93-310-311
- 10 -
